Relevance Revisited

Relevance has been well-accepted notion in the field of Information Retrieval (IR) for measuring the effectiveness of IR systems. It seems a natural choice because a typical IR system always tries to find the information that is relevant to the user’s information needs. However, this notion of relevance is not well-understood at times. Most of the IR systems use precision, recall, or some similar measure to evaluate the relevance and claim that they are able to find more relevant information due to the higher values of these measures. But the question is, do these measures that are meant for evaluating the effectiveness of the system or the algorithm, adequately reflect the relevance from the user point of view? In order to investigate this question I revisited the notion of relevance as given in the literature of various fields. It turns out that more than often we disregard the user’s point of view regarding relevance and use the relevance as measured by the systems as the selling point.